Texture synthesis involves automatically generating large textures from a small example image (known as a texture sample or exemplar). This exemplar-based texture synthesis takes an exemplar and generates additional content based on that exemplar to create much more content than is contained in the exemplar. Exemplar-based texture synthesis alleviates the need to store explicitly an entire expanse of content. Instead, the content is generated “on the fly” from the much smaller exemplar.
Traditionally, exemplar-based texture synthesis includes a correction process that compares neighborhoods of each synthesized pixel with neighborhoods of the exemplar. The synthesized pixels then are modified to recreate synthesized neighborhoods that are compatible with the exemplar content. For each pixel, the best matching neighborhood is found, and then the current pixel is replaced with the best-matching pixel. The best-matching neighborhood is determined by comparing pixel colors in a small grid of pixels. For example, for each pixel, a 5×5 neighborhood of pixels may be examined, and the error of the neighborhood is the sum of the errors of the red, green, and blue (RGB) color vectors of individual pixels in the 5×5 neighborhood.
An important application for exemplar-based texture synthesis is the generation of surface texture. Some types of surface texture synthesis include a zero velocity field, whereby the texture is stationary on the surface. Other types of surface texture synthesis techniques include synthesizing texture in space-time with a non-zero velocity field. This gives the appearance of the texture “flowing” over the surface over time. This type of non-zero velocity field texture synthesis is called texture advection.
Texture advection utilizes an exemplar and a velocity field. The velocity field is a collection of vectors specified over a synthesized image. Specifically, the velocity field specifies a velocity vector for each pixel in the synthesized image. Applications for texture advection include texture-based flow visualization and textured animated fluids, such as water, foam, or lava.
The challenge of texture advection is to maintain both spatial and temporal continuity in the advecting texture without the introduction of blurring or ghosting. In other words, it is desirable to have each frame of the animation have the quality of a still synthesized image. Thus, the challenge in texture advection is to move the texture according to a velocity field, while also having the texture in any time frame look visually similar to the given exemplar texture.
One texture advection technique blends several advecting layers of texture that are regenerated periodically out-of-phase, and reduces ghosting by adapting the blend weights to the accumulated texture deformation. Another technique uses the color synthesis of a previous time frame as a soft constraint. In particular, the technique runs an optimization process to do synthesis, and the goal is to make the current synthesis results be as close as possible to the predicted result from the previous time frame. In order to synthesize the next frame, the previous result is warped using the vector field and used as a soft constraint to the texture synthesis optimization. If only the previous result are warped using the vector field, a texture is obtained where the texture elements can be deformed. This deformation becomes progressively worse over time. This prediction is used as a soft constraint for synthesis so that the synthesis can recreate a “clean” texture with undeformed texture elements. Thus, the use of a soft constraint encourages the newly synthesized texture to be aligned with the texture advected from the previous frame.
One drawback to the last technique is that it requires use of a complicated optimization process. Moreover, there is the possibility of coherence loss. Coherence loss can occur because the optimization process is started from scratch in the current time frame, and then tries to rematch the new constraint.